Papers with detection accuracy

40 papers
SCRIPTMIND: Crime Script Inference and Cognitive Evaluation for LLM-based Social Engineering Scam Detection System (2026.eacl-industry)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown promise in identifying deception, but their cognitive assistance potential remains underexplored.
Approach: They propose a framework for LLM-based scam detection that bridges automated reasoning and human cognition.
Outcome: The proposed framework outperforms GPT-4o in the Korean scam detection and phone scam simulations.
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better (2023.acl-long)

Copied to clipboard

Challenge: a recent study shows that without artificially encouraging models to hallucinate, existing methods fall short . hallucinations are cases when the model generates output that is partially or fully unrelated to the source sentence.
Approach: They propose a method that evaluates the percentage of the source contribution to a generated translation.
Outcome: The proposed method improves detection accuracy for the most severe hallucinations by a factor of 2.
Improving Hate Speech Detection by Fusing Textual and User Interaction Representations in Online Communities (2026.acl-industry)

Copied to clipboard

Challenge: Existing studies on toxic content in online communities are limited by the scarcity of data that align textual content with comprehensive social interactions.
Approach: They propose a user-aware hate speech detection framework that effectively fuses textual semantics with social interaction representations to provide pragmatic context for disambiguation.
Outcome: The proposed framework outperforms strong text-only baselines by over 3.6%, validating the critical role of social context in enhancing detection accuracy.
Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge (2025.acl-long)

Copied to clipboard

Challenge: Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features .
Approach: They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction.
Outcome: The proposed framework outperforms baseline methods on four datasets.
Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation (2025.findings-emnlp)

Copied to clipboard

Challenge: Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on halluciation detection remains underexplored.
Approach: They conduct an empirical evaluation of CoT prompting in Large Language Models (LLMs) to examine their impact on hallucination detection methods.
Outcome: The proposed method significantly affects the internal states and token probability distributions of the LLM.
Unsupervised Out-of-Domain Detection via Pre-trained Transformers (2021.acl-long)

Copied to clipboard

Challenge: Prior work on out-of-domain detection requires in-domain task labels and is limited to supervised classification scenarios.
Approach: They propose a method to construct out-of-domain detectors efficiently using pre-trained transformers.
Outcome: The proposed method greatly improves out-of-domain detection ability in a more general scenario.
WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents (2025.findings-naacl)

Copied to clipboard

Challenge: Existing methods focus on distinguishing fully watermarked text from non-watermarked text, overlooking real-world scenarios where LLMs generate only brief segments within longer documents.
Approach: They propose a method to detect watermarked segments in large documents using an anomaly extraction method and a local traversal.
Outcome: The proposed method achieves a superior balance between detection accuracy and computational efficiency.
Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing detectors use classifier-style probability signals or rely on rewriting, which can degrade quality and introduce new triggers.
Approach: They propose to efficiently remove poisoned examples before or during fine-tuning .
Outcome: The proposed method outperforms prior detectors on two machine translation datasets and one QA dataset.
RESF: Regularized-Entropy-Sensitive Fingerprinting for Black-Box Tamper Detection of Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods for tamper detection rely on model stability, not inherently stochastic models.
Approach: They propose a hypothesis-testing method for black-box tamper detection for LLMs . they propose regularized entropy-sensitive fingerprinting to enable efficient fingerprinting .
Outcome: The proposed method achieves 98.80% detection accuracy under challenging conditions . it is based on a first-order surrogate for KL divergence to identify prompts most responsive to parameter perturbations.
What Did You Learn To Hate? A Topic-Oriented Analysis of Generalization in Hate Speech Detection (2023.eacl-main)

Copied to clipboard

Challenge: Hate speech detection datasets often use different annotation guidelines, resulting in inconsistencies . authors propose a topic-oriented approach to study generalization across popular hate speech datasets .
Approach: They propose a topic-oriented approach to study generalization across popular hate speech datasets . they compare Transformer-based models in capturing topic-generic and topic-specific knowledge .
Outcome: The proposed approach improves the reliability of hate speech detection on social media platforms.
FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adapter (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for detecting fake news videos fall short due to lack of knowledge to verify the news is real or not.
Approach: They propose a VLM-based framework for detecting fake news on short video platforms . they design four experts tailored to handle each scenario and integrate them into VLM .
Outcome: The proposed framework outperforms current state-of-the-art models on two benchmark datasets.
SenDetEX: Sentence-Level AI-Generated Text Detection for Human-AI Hybrid Content via Style and Context Fusion (2025.emnlp-main)

Copied to clipboard

Challenge: Text generated by Large Language Models (LLMs) now rivals human writing, raising concerns about its misuse.
Approach: They propose a framework for sentence-level AI-generated text detection via style and context fusion.
Outcome: The proposed framework outperforms baseline models in detection accuracy while exhibiting transferability and robustness.
Cross-Domain Audio Deepfake Detection: Dataset and Analysis (2024.emnlp-main)

Copied to clipboard

Challenge: Existing audio deepfake detection datasets are outdated and lack generalization capabilities.
Approach: They construct a new cross-domain audio deepfake detection dataset comprising over 300 hours of speech data that is generated by five advanced zero-shot TTS models.
Outcome: The proposed models achieve 4.1% and 6.5% error rates in the cross-domain ADD dataset generated by five advanced zero-shot TTS models.
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs (2026.findings-acl)

Copied to clipboard

Challenge: prevailing taxonomies neglect robustness and honesty, yielding safer-on-paper but less useful systems.
Approach: They propose a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference.
Outcome: The proposed model maintains safety while reducing over-refusal.
What’s Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previews (2026.acl-long)

Copied to clipboard

Challenge: Existing efforts to detect factually incorrect content are omitted by creators who subtly reshape impressions by omitting crucial background context.
Approach: They propose a multi-stage pipeline that simulates preview-based and context-based understanding and a OMGuard pipeline that combines interpretation-aware fine-tuning and rationale-guided misleading content correction.
Outcome: The proposed framework lifts an 8B model’s detection accuracy to the level of a 235B LVLM while delivering stronger end-to-end correction.
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies focus on evaluating large language models' ability to handle disagreement cases.
Approach: They evaluate the performance of large language models in detecting offensive language at varying levels of agreement.
Outcome: The proposed model improves detection accuracy and model alignment with human judgment by using disagreement samples in training.
LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have become central to NLP, demonstrating their ability to adapt to various tasks through prompting techniques.
Approach: They probe the hidden layers of Large Language Models to identify where sentiment features are most represented and to assess how this affects sentiment analysis.
Outcome: The proposed approach enables sentiment tasks to be performed with memory requirements reduced by an average of 57%.
Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for detecting text fragments are not suitable for contract discovery, since it requires manual definition of a few examples, followed by conventional information.
Approach: They propose a task where legal clauses are extracted from documents, given a few examples of similar clauses from other legal acts.
Outcome: The proposed task differs substantially from conventional NLI and shared tasks on legal information extraction.
CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detection (2025.findings-acl)

Copied to clipboard

Challenge: Multimodal large language models have demonstrated impressive capabilities in visual reasoning and text generation.
Approach: They propose a multimodal large language model that captures deeper relationships between images and text . they propose CMIE, which uses a Coexistence Relationship Generation strategy and an AS mechanism to detect misinformation.
Outcome: The proposed framework outperforms existing methods in detecting out-of-context misinformation.
SpeechT-RAG: Reliable Depression Detection in LLMs with Retrieval-Augmented Generation Using Speech Timing Information (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have been extensively utilized for health-related tasks, yet their performance in depression detection remains limited when relying solely on text input.
Approach: They propose a system that leverages speech timing features for depression detection and reliable confidence estimation.
Outcome: The proposed system outperforms text-based RAG systems in depression detection and confidence estimation.
Text Classification by Contrastive Learning and Cross-lingual Data Augmentation for Alzheimer’s Disease Detection (2020.coling-main)

Copied to clipboard

Challenge: Existing methods for AD detection are too expensive and time-consuming to cover all potential patients.
Approach: They propose a contrastive learning method to obtain effective text representations based on monolingual embeddings of BERT and a cross-lingual data augmentation method by building autoencoders to learn the text representation shared by both languages.
Outcome: The proposed method outperforms other methods on a Mandarin AD corpus and achieves 81.6% detection accuracy.
A Survey on Detection of LLMs-Generated Content (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance .
Approach: They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy.
Outcome: The proposed model will be able to detect human-written content in real time.
RedHerring Attack: Testing the Reliability of Attack Detection (2025.emnlp-main)

Copied to clipboard

Challenge: In response to adversarial text attacks, attack detection models have been proposed and shown to successfully identify text modified by adversaries.
Approach: They propose to modify a text to cause the detection model to predict an attack while keeping the classifier correct.
Outcome: The proposed model drops detection accuracy between 20 - 71 points while maintaining (or improving) classifier accuracy.
Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Prediction (2025.acl-long)

Copied to clipboard

Challenge: Existing methods focus excessively on detection accuracy, neglecting the societal risks posed by high false positive rates (FPRs).
Approach: They propose a Conformal Prediction framework that constrains the upper bound of false positive rates and introduces a real-time detection framework.
Outcome: The proposed framework reduces false positive rates and improves detection performance.
LEAF: Towards Lightweight Explainable Hateful Video Detection via Self-Grounding CoT Guided Stage-Wise Distillation (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for detecting hateful videos rely on opaque models with no insight into their decisions.
Approach: They propose a lightweight, explainable video detection framework that distills "explainability" from LMMs into efficient Smaller Multimodal Models (SMMs) they use a self-grounded chain-of-thought mechanism to generate unbiased supervision signals for videos .
Outcome: The proposed framework outperforms existing methods in detection accuracy and explainability on three video benchmarks.
Low-Entropy Watermark Detection via Bayes’ Rule Derived Detector (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark.
Approach: They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence.
Outcome: The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks.
Unraveling the Mystery of Artifacts in Machine Generated Text (2022.lrec-1)

Copied to clipboard

Challenge: Recent studies show that human-written text is not distinguishable from synthetic text because of semantic errors or logical contradictions.
Approach: They propose to analyze the forms of artifacts left by neural Text Generation Models by corrupting texts and replacing them with linguistic or statistical features.
Outcome: The proposed method replaces text with linguistic or statistical features and improves the accuracy of the model.
What You Read Isn’t What You Hear: Linguistic Sensitivity in Deepfake Speech Detection (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in text-to-speech technology have enabled highly realistic voice generation, fueling deepfake attacks.
Approach: They propose a framework for transcript-to-audio perturbation anti-spoofing that incorporates linguistic variation into detectors to investigate spoof detection.
Outcome: The proposed framework can bypass commercial detectors by incorporating linguistic variation into the design of anti-spoofing systems.
MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation approaches focus on mLLMs’ detection accuracy for binary classification tasks, which often fail to reflect the in-depth interpretive nuance of harmfulness across diverse contexts.
Approach: They propose an agent-based arena-style evaluation framework that provides context-aware and unbiased assessment for mLLMs’ understanding of multimodal harmfulness.
Outcome: The proposed framework reduces evaluation biases of judge agents and provides unbiased comparisons of mLLMs’ abilities to interpret multimodal harmfulness.
Breaking the "Provable Security": Detecting Finite-Precision Artifacts in LLM-based Steganography via Low-Probability Vanishing (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in Large Language Models have fostered a new class of generative linguistic steganography, claim “provably secure” by theoretically aligning the stego distribution with the language model’s natural distribution.
Approach: They propose a framework that transforms the detection task from semantic classification to a statistical audit of the sampling mechanism.
Outcome: The proposed framework breaks the security of AC and Meteor with high detection accuracy, whereas state-of-the-art semantic steganalyzers degrade to random guessing.
Defending against Indirect Prompt Injection by Instruction Detection (2025.findings-emnlp)

Copied to clipboard

Challenge: Indirect Prompt Injection attacks can be exploited by LLMs that are embedded with external data.
Approach: They propose a detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.
Outcome: The proposed approach reduces the success rate of attacks to 0.03% on the BIPIA benchmark.
BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers? (2026.acl-long)

Copied to clipboard

Challenge: Existing evidence suggests that LLMs are not able to detect scientifically unsound work from malicious or poorly designed research agents.
Approach: They develop a framework that evaluates whether fabrication-oriented paper generation agents can deceive multi-model LLM review systems.
Outcome: The proposed framework shows that fabricated papers achieve acceptance rates up to 18% . the framework shows only marginal improvements, with detection accuracy barely exceeding random chance.
MedForge: Interpretable Medical Deepfake Detection via Forgery-aware Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Existing defenses against forgery are inadequate for healthcare.
Approach: They propose a large-scale benchmark for pre-hoc, evidence-grounded medical forgery detection using a doctor inspection guideline and gold edit locations.
Outcome: Experiments show that the proposed solution can detect and explain medical scans with high fidelity and accuracy.
Generate First, Then Sample: Enhancing Fake News Detection with LLM-Augmented Reinforced Sampling (2025.acl-long)

Copied to clipboard

Challenge: Existing models have a performance gap of 20% between classifying fake news and real news, making them less suitable for practical deployment.
Approach: They propose to adopt an LLM to generate fake news in three different styles, which are later incorporated into the training set to augment the representation of fake news.
Outcome: The proposed model achieves state-of-the-art performance on two benchmark datasets and improves detection accuracy by 24.02% and 11.06% respectively.
TABARD: A Novel Benchmark for Tabular Anomaly Analysis, Reasoning and Detection (2025.findings-emnlp)

Copied to clipboard

Challenge: a new benchmark is constructed to evaluate the accuracy of large language models for tabular data . the benchmark uses direct, indirect, and Chain-of-Thought prompting .
Approach: They propose a framework that uses prompting, self-verification and constraint-based rule execution to improve accuracy.
Outcome: The proposed framework significantly improves accuracy and recall in tabular data.
You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking (2026.acl-long)

Copied to clipboard

Challenge: Existing sequential LLMs cannot be directly applied to DLMs, as their generation order is arbitrary.
Approach: They propose a stability-aware constraint that allows watermarking only in stable contexts and a bit-controlled, unbiased modulation to preserve the original DLM output distribution.
Outcome: The proposed scheme achieves stable watermarking with minimal quality impact while maintaining high detection accuracy and multi-bit capacity.
Summary Factual Inconsistency Detection Based on LLMs Enhanced by Universal Information Extraction (2025.findings-acl)

Copied to clipboard

Challenge: Recent studies have shown that Large language models can detect factual inconsistencies in summaries but they lack the efficiency and explainability needed to be effective.
Approach: They propose to decouple LLMs’ information extraction and reasoning capabilities to address key challenges and propose a framework for UIEFID to guide fine-tuned LLM methods in extracting unified structured information from documents and summaries.
Outcome: The proposed framework improves the detection accuracy and reduces redundant reasoning on the AGGREFACT benchmark.
TempParaphraser: “Heating Up” Text to Evade AI-Text Detection through Paraphrasing (2025.emnlp-main)

Copied to clipboard

Challenge: Existing detectors that perform well on benchmark datasets have weaknesses that can be exploited to manipulate AI-text.
Approach: They propose a framework that simulates high-temperature sampling effects through multiple normal-temperaturing generations, effectively evading detection.
Outcome: The proposed framework reduces detector accuracy by an average of 82.5% while preserving high text quality.
Perception, Understanding and Reasoning: A Multimodal Benchmark for Video Fake News Detection (2026.acl-long)

Copied to clipboard

Challenge: Existing video fake news detection benchmarks focus on the detection accuracy, while failing to provide fine-grained assessments for the entire detection process.
Approach: They propose a process-oriented video fake news detection benchmark that evaluates MLLMs' perception, understanding, and reasoning capabilities in VFND.
Outcome: The proposed model achieves sota performance on video fake news detection tasks.
RV-HATE: Reinforced Multi-Module Voting for Implicit Hate Speech Detection (2026.acl-long)

Copied to clipboard

Challenge: a new framework for hate speech detection addresses implicit hate speech by tailoring the detection process to dataset-specific attributes.
Approach: They propose a framework to account for the dataset-specific characteristics of hate speech datasets.
Outcome: The proposed framework improves detection accuracy and provides interpretable insights into the distinctive features of each dataset.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations